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Abstract 

Applied Research (AR) in Canadian community colleges is driven by a 
mandate, via the collective voice of Colleges and Institutes Canada - a 
national voluntary membership association of publicly supported colleges 
and related institutions - to address issues of interest to industry, 
government, and/or community. AR is supported through significant federal 
and provincial level funding mechanisms as well as funding from the private 
sector. Performance measurement tools have largely been developed by 
government agencies such as the Natural Sciences and Engineering 
Research Council (NSERC) and the National Research Council of Canada 
(NRC), which are external to the colleges that engage in AR. This paper 
presents an overview of AR in Canadian community colleges and institutes 
and provides recommendations for the development of sustainable 
performance measurement frameworks for AR in Canadian community 
colleges and institutes. 


Applied Research in the Canadian Community College 
System 

Canada is home to a network of over 150 colleges and institutes that 
originally developed as a result of federal legislation and financial support 
in the 1960s in order to foster Canada’s transition to an industrial economy. 
This aforementioned group of institutions is represented by Colleges and 
Institutes Canada (formerly known as Association of Canadian Community 
Colleges), a voluntary membership association of colleges and related 
institutions. This shift occurred primarily through certificate and diploma 
level trades and other skills-based training with a direct connection to 
industry competency requirements (Madder, 2006; Fisher, 2009). 

In recent years, Canada’s community colleges and institutes have 
positioned themselves to respond to the emerging knowledge economy 
through various means, including active advocacy and support for an 
Applied Research agenda (ACCC, 2010). Applied Research (AR) involves 
the generation of new knowledge, like basic research, but is client-driven in 
that it seeks to address matters of direct practical relevance to industry, 
community, or government (Roll-Flansen, 2009; OECD, 2002). 

Colleges and Institutes Canada (CIC), mentioned above, has been 
promoting Applied Research since 2002 (Madder, 2006). AR in the 
Canadian college system has developed substantially from its humble 
beginnings at the turn of the 21st century with little activity and virtually no 
formal support. By 2012, nearly 100 institutions had dedicated AR divisions 
with almost 5,000 industry and community research partnerships, over 




1,500 faculty and staff working on AR projects, roughly 24,000 students 
participating in AR activities, and $179,000,000 in external funding (ACCC, 
2012 ). 

ACCC (2011) constructed an applied research college typology based 
on the state of development of their applied research activities, including 

• institutions with neither applied research activities nor 
governance mechanism; 

• novice innovation institutions; 

• established innovation institutions; and 

• integrated innovation institutions. 

Federal support, through the College and Community Innovation (CCI) 
Program, has played a significant role in the enhancement of AR activities 
in the Canadian community college system. The CCI program was piloted 
in 2002 and made permanent in 2008. This competitive granting program is 
administered by the Natural Sciences and Engineering Research Council 
(NSERC) on behalf of NSERC, the Social Sciences and Humanities 
Research Council (SSHRC), and the Canadian Institutes of Health 
Research (CIHR). CCI’s mandate is to support colleges to leverage their 
existing relationships with SMEs (Small and Medium Sized Enterprises) in 
order to address innovation related challenges faced by those SMEs, 
including AR, commercialization, technology transfer and new technology 
adaptation/adoption (ACCC, 2013). The CCI budget has increased from 
approximately $15,000,000 in its inception year of 2008 to just over 
$30,000,000 in 2013 (Deyirmendjian et al., 2013). Other federal sources of 
funding for college level innovation include the Canadian Foundation for 
Innovation and Canada’s National Research Council. In addition, various 
provincial and regional sources of funds support applied research in 
community colleges. 

Performance Measurement in Public Sector Organizations 

Public sector performance measurement is a universal expectation 
across jurisdictions and at different levels of government in both Canada 
and the US. Scholars trace the beginnings of Canadian performance 
measurement back to initiatives designed to improve productivity in 
American cities at the turn of the 20th century (McDavid & Hawthorn, 

2006). Planning, programming, and budgeting systems (PPBS) were 
incorporated into public sector governance in the 1960s and 1970s. This 
emphasis on accountability declined in popularity through the 1980s, 
however, a resurgence in interest in performance measurement occurred 
during the development of New Public Management (NPM) in the 1990’s, 
with an increased use of sophisticated private sector approaches to 
governance. Since then, performance measurement has been used as a 
means to facilitate effective public sector management in a number of 
western countries (McDavid & Hawthorn, 2006). 

Performance Measurement in Community College Applied 
Research 

College research that is funded by the CCI programme requires 
reporting on a number of NSERC mandated performance indicators. 
NSERC has 6 grant types within their CCI programme and of those, only 3 



require performance reporting: the Industrial Research Chairs for Colleges 
programme (12 indicators), the Innovation Enhancement grant (24 
indicators), and the Technology Access Centre grant (9 indicators) (St 
Pierre, 2013). This is the primary, if not only, performance reporting 
requirement common to applied research at Canadian colleges and its 
purpose is primarily summative. An evaluation of NSERC’s CCI program by 
Goss Gilroy (2012) revealed that performance reporting requirements were 
onerous and lacking in guidance regarding level of detail required. Luke 
and Van den Berg (2013), in a recent presentation at the last annual ACCC 
Applied Research Symposium in Quebec City, indicated that Canadian 
colleges and NSERC are exploring ways to improve and enhance 
performance reporting requirements for applied research activities. 

Applied Research Performance Measurement System 
Sustainability 

Perhaps the perception of NSERC CCI performance reporting as 
onerous and confusing is, in part, due to the fact that the measures have 
been developed by an external agency (NSERC) for summative (e.g., 
accountability) purposes. 

Dual-purpose (formative and summative) performance measurement 
systems, although uncommon, offer the promise of a compromise between 
utility for managers and the accountability requirements of funders. The city 
of Lethbridge, Alberta, has balanced the uses of their performance 
measurement system for both formative purposes, such as internal 
management of process, and for summative purposes such as external 
reporting to council (Hildebrand & McDavid, 2011). The authors also delimit 
a set of conditions that facilitate more participatory approaches to 
performance measurement in local government, some of which are of 
relevance to AR performance measurement systems in Canadian colleges 
and institutes (Hildebrand & McDavid, 2011), including: 

• Institutional size - the medium- to larger-sized 
governments (and in this case institutions) have more 
person power to implement performance measurement 
systems with dual purpose. Careful choice of indicators, 
data collection techniques, and reporting will be 
necessary to ensure that the process is feasible for 
small, medium, and large size Canadian colleges. 

• Polarization - the more polarized a given political culture, 
the more difficult it will be to implement a dual purpose 
performance measurement system since performance 
results may - or may be perceived to - be used for 
punitive purposes. From personal experience, the author 
has noticed that faculty stakeholders tend to conflate 
institutional or program level performance measurement 
with individual performance appraisal which can, 
depending on the organizational culture of the given 
college, create reluctance to participate in an applied 
research performance measurement initiative. Faculty 
trepidation may be minimized if managers clearly 
differentiate AR performance measurement activities 
from individual performance and if the activity is explicitly 
non-punitive. 

• Bottom-up approach - dual purposed systems are more 
likely when program managers and other stakeholders 
(key users of the performance measurement system) are 



involved in designing and implementing the system. 


Another study by Pollitt et al. (2010), on performance regimes in health 
care systems, cautions against several negative trends that occur over time 
within a given performance measurement system. These include: 

• Increase in the number of performance indicators - 
performance measurement systems tend to start with 
relatively few indicators that expand in number to 
broaden the system’s scope of coverage, the results of 
which are often merged into complicated aggregate 
measures that tend to erode public confidence in the 
system. An increase in the number of indicators may 
make the system impractical to implement, however 
additional indicators may need to be added as new 
research clusters are incorporated into a given college’s 
applied research activities. 

• Shift from a formative approach to a summative 
approach - over time, performance indicators shifts from 
formative uses for internal improvement to externally or 
publically available summative uses (e.g., ranking 
systems or league tables). Summative uses tend to 
engender gaming the measures to avoid unwanted 
consequences of failure in high-stakes contexts. 

Stakeholders involved in AR in the college system will 
have to be on guard for performance measurement 
system usage pattern shifts - particularly as the 
programme increases in profile and public recognition - 
and be prepared to advocate for the continuance of a 
dual purpose performance measurement system. 

Future Directions for AR performance measurement in 
Canadian Colleges 

Effective performance measurement of applied research activities in 
the college system have the potential to contribute greatly to both the 
efficiency and effectiveness of AR programs which would ultimately provide 
direct benefit to community college’s key stakeholders: students and 
regional employers. The following recommendations could help to improve 
the utility and functionality of community college AR performance 
measurement systems: 

• Borrow performance evaluation frameworks and specific 
measures from university research, private sector 
innovation , and non-profit fields. The performance of 
university research programs has typically been 
measured via peer review and bibliometric approaches 
(Geuna & Martin, 2003). Performance measurement of 
private sector innovation activity is more wholistic in that 
it typically involves measurement at one of more of the 
recognized stages of the innovation cycle including idea 
generation, concept development, concept evaluation, 
development, and implementation (du Preez & Louw, 

2008). Non-profit performance measurement 
frameworks are typically as diverse as the non-profits 
that use them (Eckerd & Moulton, 2011), however many 
focus on both the process and the impacts of the work 
on the beneficiary communities. To date, most 
community college research has emphasized 
technological innovation (ACCC, 2013). Measures from 
the private sector innovation field are most appropriate 
for this type of community college applied research. As 



more community colleges delve into social innovation 
research, the frameworks and measures used by 
nonprofits will become increasingly relevant. University 
research performance measures have the least 
relevance for community college applied research 
programs however peer review and bibliometric 
measures could have ancillary value as indicators of 
contributions to the broader applied research academic 
community. 

• Use a mix of formative and summative approaches. 
Community colleges can use AR performance evaluation 
as both an organizational learning tool and a means to 
report on effectiveness and impact. ACCC (2011) 
described a typology of AR institutions along a 
continuum ranging from community colleges with no 
formal innovation governance structures to those with an 
innovation agenda that is integrated into multiple levels 
of the organization and the local economy. An adaptive 
approach to performance evaluation of community 
college AR would include a greater emphasis on 
formative measures and approaches at the early 
innovation stages as a means to foster institutional 
capacity building and therefore movement along the 
continuum. 

• Pair institutional research tools and expertise with the 
needs of community college applied research offices. 

Most community colleges have dedicated institutional 
research staff. Ensuring that applied research 
performance measurement systems are compatible with 
institutional research frameworks could facilitate 
leveraging the aforementioned expertise for performance 
data collection and analysis and greater integration of 
AR into broader institutional planning initiatives. 

• Establish a national community college applied research 
performance evaluation working group. Currently, 
performance measures in use for community college AR 
are determined by the primary funder, the Natural 
Sciences and Engineering Research Council (NSERC). 
Externally mandated and generated performance 
measures are not uncommon, but have led to 
documented cases of ‘gaming’ measures (Pollitt et al., 

2010) in order to avoid possible negative repercussions 
from donors. A participatory approach to performance 
tool development, implementation, and refinement would 
result in greater ownership on the part of the colleges 
and other stakeholders and would allow for the 
generation and dissemination of best practices around 
AR performance measurement and management. This 
would consequently result in a more active and authentic 
use of the performance measurement system. Such a 
performance evaluation working group should consist of 
an array of stakeholders including college institutional 
and applied research administrators, research faculty, 
students, industry/employer partners, and 
representatives from funding agencies. 

Conclusion 

A sustainable performance measurement framework for AR in 
Canadian community colleges and institutes will borrow proven 
performance measurement elements from the public, private, and non-profit 
sectors. Coordination with internal institutional research divisions will 



ensure that AR is consistent with a given college’s strategic direction. A 
context-dependent mix of formative and summative approaches and direct 
involvement of college AR staff in the development of the performance 
measurement framework will also contribute to the long-term utility and 
sustainability of a given framework. Finally, the establishment of a national 
community college AR performance measurement working group will help 
to both engender ownership of these performance tools, and encourage a 
national community of practice. 
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